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Abstract: Globalisation and international mobility in the 21st century has led to the 
internationalisation of the English language (Crystal, 2003). Research regarding linguistic gains 
at university levels is however extremely scarce. This study aims to address this gap of 
knowledge and provide some answers as to how much linguistic gain can be expected after one 
year of English medium instruction. Two groups of undergraduate students enrolled in different 
levels of English medium instruction (EMI) were given a pre and post-test over a 1-year period. 
Results were analysed statistically; significant gains were found only in the semi- immersion 
group in the grammatical domain; although, there was a trend for improvement as well as higher 
scores for full immersion students. It might be interpreted that in order for linguistic gains to be 
seen in adults there needs to be some focus on form and language guidance (Munoz, 2007; 
Perez-Vidal, 2007). Thus, an integrated content and language (ICLHE) approach is more 
effective than a solely content based EMI model for university level content courses, if linguistic 
gains are the desired outcomes of the programme. 
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Introduction 

This preliminary study examines the effects of English medium instruction (EMI). The 
article supplies an overview of the rise of English as a Lingua Franca in Higher Education (HE), 
focusing in particular on the European Union setting and the policies that support 
multilingualism in education. To set the background to the study, two fields are reviewed. Firstly, 
Content and Language Integrated Learning (CLIL), which is considered a predecessor of EMI, 
because it has been in place for some years across primary and secondary education levels, as 
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well as across European countries. Secondly, research from Second Language Acquisition 
(SLA) is discussed to highlight the importance of periods of intense exposure in SLA 
development. The emerging field of Integrated Content and Language in Higher Education 
(ICLHE) is outlined, providing the distinction between ICLHE and EMI. Existing qualitative 
research describing EMI at different European universities is provided, including perspectives 
from both lecturers and students. Empirical research regarding linguistic gains in EMI students 
is presented before describing the current exploratory study. 

English in Higher Education and the European Union 

It is estimated that approximately 400 million of the world’s population speak English as 
their native language; while more, approximately 430 million, speak it as their second language 
(L2). The British Council estimated one billion people were studying English as a foreign 
language in the year 2000 (Crystal, 2003). English is often the first choice when choosing to 
learn a second language. English is commonly used among professionals who do not speak the 
same first language, thus giving it the status of a Lingua Franca (House, 2013). The driving 
force behind the rise of English as an international language is globalisation, mobility, and 
international business. The ability to communicate in English is an asset, allowing professionals 
to work in the international context, leading to increased career opportunities as well as higher 
earning capacity. University students in Spain identify one of the reasons they want to study in 
English as “feeling the need to have the linguistic and cultural skills to function effectively in an 
international social context and labour market” (Fortanet-Gomez, 2013). The rise of English in 
the stated context goes hand in hand with English being used as the dominant language of 
research in HE in Europe and around the world. 

The European Union set out a new agenda with regard to multilingualism and 
interculturalism in the educational sphere in 1999: The Bologna Process was released with the 
goal to develop a coherent and cohesive European higher education arena (EHEA). This would 
ease mobility and hiring for employers and professionals, as well as make university exchanges 
more possible for students. This process encouraged universities to attract international 
students, to develop economic and cultural collaborations with other countries including the 
mobility of students and faculty, and made it possible for citizens to develop intercultural skills 
(European Commission, 1999). Nine years later, the European Commission stressed the 
importance of intercultural dialogue and the need for more effort to achieve proficiency in the 
mother tongue plus two languages an objective for all citizens, in stating that “[a] key instrument 
in this respect is the Barcelona objective-communication in mother tongue plus two 
languages. More effort is needed towards achieving this objective for all citizens” (European 
Commission, 2008, p. 5, emphasis in original). 

After these policy changes, study abroad and exchange programmes such as the well- 
known Erasmus became even more popular, attractive, and attainable for students, 
encouraging them to spend part of their university period at a participating university. The rise of 
English as a Lingua Franca came about through a cyclical process, where the increased use of 
English fuelled the importance of English in HE, which in turn encouraged national policy 
changes to support the educational system, which then goes back to strengthen the position of 
English as a lingua franca (Juan-Garau & Salazar-Norguera, 2015; Perez-Vidal, 2014; Smit & 
Dafouz, 2012). This is exemplified by HE institutions reactions to English language, to which we 
will now turn. 

Universities began to offer part or whole degree programmes in languages other than 
the official or traditional languages of the country, most noticeably English (Llurda, Cots, & 
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Armengol, 2013). The mobility of citizens and the release of the Bologna process required the 
development of new language policies, whether it be to encourage or protect minority, 
neighbouring or national languages or to incorporate English or both. The first universities to 
implement changes in language instruction were mainly concentrated in the northern European 
countries, Holland, and Sweden, among others, and as time went on EMI became established 
as a trend throughout Europe. A recent survey of EMI programmes at European universities by 
Wachter and Maiworm (2008) revealed 2,400 programmes taught entirely through the English 
language. They approximate about 2% of all European students participating in these 
programmes. 

The main objectives of the internationalisation of European universities are, firstly, to 
attract the most knowledgeable scholars in the given field, and to increase the number of 
international students, which increases revenue and international recognition. Secondly, to offer 
internationalisation at home (laH) for national students, who can reap the benefits of 
international exposure while studying at their local university (Llurda et al. 2013; Strotmann et 
al., 2014; Thogersen, 2013). 

The amount of total exposure to English through EMI often varies from university to 
university. Nastansky (2004) identified three models adopted when implementing English taught 
programmes in university settings. Firstly, a programme in which the amount of English use 
remains the same throughout the programme. The second model increases the amount of 
English taught courses as the degree goes on, and the third, adopted in Germany, involves 
decreasing the amount of English taught courses as the degree goes on. 

The types of changes in university policies discussed in this section have circular effects 
which can be summarised by the following quote, “[wjhile the global status of English impels its 
adoption in higher education, the adoption of English in higher education further advances its 
global influence” (Coleman, 2006, p. 4). 

Review of the Literature 

The following sections provide a background to EMI in HE and put the study into context. 
CLIL is defined and discussed as a European approach to multilingual education that has been 
in use at primary and secondary levels for some time now. Following this, the rationale and 
development behind CLIL as an educational model is detailed, leading to a definition. Finally, a 
description and examples of ICLHE and EMI are given. The article then goes on to highlight the 
research findings from both CLIL as well as EMI. The review closes with a summary of what 
type of linguistic gains we can expect to find in EMI based on the current knowledge in the field. 

CLIL Education 

In the 1990’s, various types of multilingual education were in place throughout Europe, 
due in part to trends in internationalisation and globalization (Marsh, Pavon-Vazquez, Frigols 
Marin, 2013). Already by the late 1980’s, terminology referring to “content-based second/foreign 
language instruction” was appearing in the literature (see Brinton, Snow, & Wesche, 1989; 
Snow, Met, & Genessee, 1989). In 1994, a report from the Continuing Education Centre of the 
University of Jyvaskyla in Finland expanded on the Teaching Content through English (TCE) 
programme in that institution during 1992-1993 (Rasanen & Marsh, 1994). In that same year, 
Marsh (1994) pointed out that “[i]n CLIL the learning of language and other subjects is mixed in 
one way or another. This means that in the class there are two main aims, one related to the 
subject, topic or theme and one linked to the language” (p. 6). Then, in 1995, the European 


Linguistic Outcomes of English Medium Instruction Programmes in Higher Education 


49 





www.hlrcjournal.com 


Open 0 Access 


Commission issued a white paper, Teaching and learning: Towards the learning society, where 
multilingualism was encouraged as part of being European: 

Languages are also the key to knowing other people. Proficiency in languages helps to 
build up the feeling of being European with all its cultural wealth and diversity and of 

understanding between the citizens.it is becoming necessary for everyone, 

irrespective of training and education routes chosen, to be able to acquire and keep up 
their ability to communicate in at least two Community languages in addition to their 
mother tongue. (European Commission, 1995, p. 47). 

Three years later, Nikula and Marsh were making the case to consider content and language 
integrated learning (CLIL) the main “umbrella” term to encompass pedagogies that integrate 
language and content, in which Enqlish or another foreign language is used as the means of 
instruction (1998, p. 4-6). 

In sum, CLIL is a methodological educational tool in response to the internationalisation 
of Europe and the consideration of multilingualism as an asset, as referred to in the previous 
section; it is a multifaceted approach, encompassing social, linguistic, educational and cognitive 
levels (Perez-Vidal, 2009). 

An example of CLIL is elementary students in Switzerland receiving some of their 
courses in German or French, two of the nation’s official languages (Stohler, 2006), or Dutch 
secondary students having content courses such as History or Geography taught through 
English as a foreign language (Admiraal, Westholf, & Bot, 2006). Another example of an 
existing content learning model was in place in the Basque Country’s multilingual schools. 
These schools provided education to Basque children in their mother tongue, which had been 
reduced to a minority language due to political reasons (Cenoz, 2009). This Spanish region, 
along with Catalonia and Galicia, implemented similar approaches in the 1980s and continue to 
embrace multilingual education methodologies such as CLIL (Celaya & Perez-Vidal, 2001; 
Lasagabaster & Ruiz de Zarobe, 2010). These examples illustrate the use of CLIL not only to 
instruct a foreign language, but also to maintain and spread regional languages. 

CLIL Rationale 

Second language education has developed overtime. Trends in the field have changed 
from traditional formal instruction (FI) methods, where students are exposed to the secondary 
language for an hour or two per week, to more innovative, communicative, immersion, and 
content based methods (Richards & Rodgers, 2014). FI has been popular for many years and is 
still common practice in many countries and schools around the world. One of the main features 
of FI classrooms is that a large portion of time is spent focusing on form, providing less 
opportunity for learners to communicate and interact using the language (Richards & Rodgers, 
2014, p. 175-176). The teacher is typically the language expert and primary user, while students 
learn the language directly, often through grammar exercises. Materials used in the classroom 
are catered to the student, meaning inauthentic material. Further, the range of communicative 
contexts the learner is exposed to is limited to the classroom setting. Additionally, there is very 
limited contact with native speakers or authentic input from the target language (TL) (Perez- 
Vidal, 2011; Richards & Rodgers, 2014, p. 90). 

At the higher education level, following the revolution taking place in EFL in general with 
the communicative approach to language teaching, changes in the teaching of academic 
English emerged in England in the 1980’s and 1990’s, aiming to enhance students’ knowledge 
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and to aid them in achieving advanced levels in specific subject areas. This became known as 
English for specific purposes (ESP) and is used to address the specific language needs of the 
student (Hutchinson & Waters, 1987, p. 6-7). The approach was later adopted in many non- 
English speaking countries and eventually became known as English for academic purposes 
(EAP) (Jordan, 1997). 

This can be seen as one of the first responses to the internationalisation of universities 
and student mobility, and the beginning of the rise of English in Academia. ESP provided 
students with the tools they needed to communicate in English depending on their field, be it 
law, medicine, tourism, academics, and so on while EAP supported students studying in a 
multilingual context, especially at the tertiary level (these were usually foreign students studying 
in English speaking countries) (Fortanet-Gomez, 2013). While these models were being 
adopted in HE, it was not the only development at the time. The communicative approach 
brought both meaning and communication elements into language teaching, the adoption of this 
approach inspired changes in materials and classroom procedures at other levels of education 
(Perez-Vidal, 2009). The same concept was the rationale for the common place idea that study 
abroad stays spur linguistic development. This is supported by Munoz (2012), who concluded 
that intensive exposure may lead to a turning point in L2. For Munoz’s study, 142 undergraduate 
students in Spain were questioned, with results showing that 85% identified an intensive 
learning context, for example a study abroad, as the turning point in the quality of their English 
level. In line with these findings, Perez-Vidal (2014) found that, after a study abroad period 
(intensive input exposure), university students make gains in fluency, accuracy, and use of 
formulaic language (see also Serrano, Llanes, & Tragant, 2011; Valls-Ferrer & Mora, 2014). 
These results can be attributed to increased motivation, which is known to be an important 
factor in SLA. 

Another approach to multilingual education is content-based instruction (CBI), which 
emerged in the United States in the 1970’s and 1980’s in response to the students’ low 
proficiency English levels at all levels of education. Schools implemented courses where 
students were encouraged to think and learn in the TL. CBI instruction differs from FI in that 
language activities are always specific to the content taught, the focus changes from being not 
only on the TL but also on the actual content. Through focusing on content, learners are 
encouraged to use the language as a tool for communication rather than study about it (Snow, 
2001 ). 


Following this, a bilingual education model emerged, French immersion, developed in 
response to French becoming the official second language in Canada. The original immersion 
structure involves instructing the first three years of children’s school experience exclusively in 
French. Input then tapers off until half of the curriculum is taught in French and half in English by 
the time students are in the final year of primary school. However, over the years there has 
been many alternatives to the original system. Varying among provinces and schools, some 
start immersion in the 4th or 5th grade, while others only have a few years of intense immersion 
before returning to an English curriculum (Genesee, 2014). Findings from immersion research 
are positive and the model is considered a success (Swain, 2000). Research has demonstrated 
that while learners develop high levels of proficiency in French, they also master content 
subjects as well as their monolingual peers. Additionally, their cognitive as well as mother 
tongue development is not impeded (Shapson, 1984). Research into the linguistic outcomes of 
French immersion shows that immersion students outperform their FI counterparts on all tasks, 
and are better compared to francophone students of their grade. Thus, they tend to achieve 
near native like receptive skills, reading, and listening; however, speaking and writing skills 
remain non-native like (Swain & Lapkin,1982). 
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The development of CLIL in Europe was able to benefit greatly from and adapt research 
findings from the Canadian immersion experience, thus FonF (Focus on Form) was combined 
with a focus on communication and content. In CLIL, content subjects such as math, science, or 
history are taught through a second or foreign language. A recent definition, taken from Dalton- 
Puffer (2011), reads, “CLIL can be described as an educational approach where curricular 
content is taught through the medium of a foreign language, typically to students participating in 
some form of mainstream education at the primary, secondary, or tertiary level” (p. 183). 

There are some key characteristics of CLIL that foster linguistic benefits. Firstly, 
programmes run alongside conventional FI programmes and thus increase amount of exposure 
to the TL. Secondly, authentic material is used triggering authentic interaction. Finally, using a 
foreign language to learn content stimulates interaction, and communication becomes 
meaningful because the language is used as a tool for real communication (Perez-Vidal, 2009). 
These factors expand the number of domains and functions in which the students must use the 
TL (Escobar, 2004). 

Defining ICLHE and EMI Education 

ICLHE is considered to have the same focus as CLIL but in the HE arena. Specifically, 
language outcomes are on the agenda as well as content. Whereas, EMI is focused on content 
only and no L2 language support is given, the other obvious difference is that EMI is explicitly 
English while ICLHE is used for many other languages (Smit & Dafouz 2012). 

The distinction between ICLHE and EMI is further exemplified through the following two 
studies, one at an Italian university and the other at a Swedish university. Costa (2012) found 
Italian lecturers switch from using the language for purely communication (content) to look at it 
objectively (FonF) (Costa, 2012). For example, terms are explained in the LI or language 
structure becomes the focus. The researcher concluded that language was seen as second to 
content, and that lectures were “mainly meaning focused but with incursions into language focus 
and thus moving gradually into ICHLE” (Costa, 2012, p. 42). Whereas, in the Swedish study, 
Airey (2012) considered most programs to be EMI, as they have no focus on language. It was 
even noted that in some cases professors do not feel comfortable correcting students’ mistakes 
(Unterberger, 2012), especially concerning grammatical errors over lexical ones (Airey, 2012; 
Costa, 2012). 

ICLHE is an emerging field of research; it has grown quickly in response to the circular 
process described previously. Researchers are beginning to explore this subfield of multilingual 
education. Due to this, research to date is mainly descriptive and exploratory, much like the 
present study. Qualitative research has been carried out, examining the different models 
adopted as each country and university is a different case and adapts EMI to fit their own 
unique needs. This is why it is helpful to draw on previous quantitative studies on CLIL to 
understand language outcomes. What is interesting is that holding a university degree that was 
taught in English seems to hold value in the minds of many. It is considered an asset when 
looking for work (Perez-Vidal, 2014). Thus, by extension the general public is equating finishing 
a degree through EMI with having a high level of English proficiency. It would be interesting and 
well worth it to evaluate real linguistic outcomes of EMI. This is what the current study purports 
to do by filling this gap in research and collecting data on EMI in HE. 

Research Findings 

As mentioned previously, EMI is still a new field and it is useful to draw upon the CLIL 
experience for insight to what could be expected in the HE context. Empirical research has 
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shown that CLIL students outperform FI students in receptive vocabulary tasks (Jimenez & Ruiz 
de Zarobe, 2009). In fact, gains are noted as well in many receptive language skills, such as 
receptive vocabulary (as mentioned before), reading, speaking (in regards to fluency and risk 
taking), writing (in regards to fluency and lexical syntactic complexities), some morphological 
phenomenon, and emotive and affective outcomes. Skills that have not been shown to improve 
in CLIL environments are syntax, productive vocabulary, informal/non-technical language, 
writing (in regards to accuracy and discourse skills), and pronunciation (Ruiz de Zarobe, 2011). 

The positive effects of CLIL on linguistic skills are often attributed to the increased 
exposure to the TL, as SLA principles described above support. Linguistic gains are reported 
after providing learners with intense periods of increased input rather than spacing out exposure 
over longer periods. Investigating English immersion children in Quebec, Spada and Lightbown 
(1989) found the group exposed to an intensive block of hours performed better and were more 
advanced on all tasks when compared to a group exposed to a shorter amount of hours over a 
longer period. Intensive language programs for young learners also support these findings. 
Housen (2012) studied 105 Dutch, French, and Greek speaking students across 5-year levels, 
including primary and secondary school in the European school in Brussels. Results showed an 
increase of development at the point of increased input in this case in both intensive and formal 
language instruction classes. In a second study, on English L2 proficiency among 72 Italian LI 
students in year 3 and 4, in three different European schools in Italy, Belgium, and the UK, 
demonstrated that English extracurricular activities provided more contact with the TL and the 
additional L2 input made a significant difference in fluency, grammar, and lexicon. Studies on 
adults follow the same trend. Serrano and Munoz (2007) found gains in all linguistic domains as 
measured in a group exposed to intensive classes (25 hours per week), compared to a group 
exposed to just 4 hours of instruction per week. 

As seen, gains accrue in children and adolescents’ linguistic levels when levels of input 
are increased. Thus, it could be expected to see similar gains in university students’ linguistic 
skills. In fact, students who elected to study their university courses in English reported 
perceived linguistic gains as one of the motivating factors for enrolling in EMI (Tazl, 2011). 
However, there has been little empirical evidence to support this claim. Existing research paints 
a picture of what exactly EMI is and how stakeholders are reacting and adapting to it. Some 
academic fields find EMI to be more natural than others. In a study of Physics professors in 
Sweden, Airey (2012) found lecturers preferred teaching in English and even felt English is the 
professional language for physicists. This is echoed in Unterberger (2012), where German 
professors surveyed also found English to be the natural choice for instruction in the 
Engineering field. In the same survey, some reported EMI lectures were easier than LI lectures. 

Students perceived more gains than losses in studying in an additional language, 
according to a questionnaire involving 93 Hebrew and Arabic students studying through English 
and 47 Russian and Arabic students studying through Hebrew (Smith, 2004). In a survey at a 
Belgian university that encompassed the view points of staff and students of five different 
faculties, a total of 627 participants found that lecturers favored EMI while students preferred to 
study in their LI. Both groups felt that their English skills improve as a by-product of teaching or 
studying in English; although, students placed more emphasis on this, as they hoped and 
expected to improve their language skills, which echoes Smith’s findings above. This caused 
them to expect native-like lecturers, whereas lecturers focus on content and accept far from 
native-like levels from students, provided that communication is not hindered (Sercu, 2004). 

Most professors report more prep time and a harder workload when teaching in English 
(Airey, 2011; Tazl, 2011). Impressions from students showed that in the first year of taking EMI 
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courses, they found the workload much harder and felt they would perform better if the courses 
were in their native language. However, professors felt the students struggled more with content 
than language (Tazl, 2011). 

Turning to the focus of this study, there is reason to believe that students’ linguistic 
levels will improve when enrolled and participating in EMI courses. All students in EMI 
programmes have studied English before they enter the programme; due to this factor, starting 
EMI can be considered a period of increased exposure to the target language. Increased 
exposure has been shown to lead to gains in language abilities in both children and 
adolescents. This leads us to expect some linguistic improvements in HE students; more 
noticeable gains are expected in receptive language tasks, and less gains in writing and 
grammar tasks as found in CLIL and immersion learners. As far as could be established, only 
one study exists examining the development of linguistic abilities after an EMI treatment. 
Loranc-Paszylk (2007) measured undergraduate students’ performance on a reading task. Two 
groups (N=39) differed in both the type of language instruction they received as well as the 
amount of EMI exposure. The first was an ICLHE programme in International Relations and the 
second an English Philology programme (FI, FonF). The task used was the CAE reading 
proficiency test. The author concluded that, even though the ICLHE group had 60% less 
exposure to English, the performance of each group was comparable. The author concluded 
that reading skills acquired through just one discipline are transferable and help L2 performance 
overall. For example, when students chose to write their final projects in English, those who had 
higher reading scores performed better in their writing tasks. 

In summary, the study outlined above points to linguistic gains on reading tasks by 
ICLHE students. However, in the current review of research in EMI there seem to be no studies 
examining linguistic gains based on the performance of lexico-grammatical, listening, and 
writing tasks, even though there is a perceived value in EMI. More research is needed in order 
to support these perceptions. Given that these questions remain unanswered, this study aims to 
address them. 


Research Questions 

This preliminary study examines linguistic gains measured through four tasks: an oral 
comprehension (listening) task, a written composition, a cloze task, and a grammar task. The 
research questions are: 

1. Do students who take 100% of their degree through EMI experience linguistic gains 
over a one-year period? 

2. Do students who take 50% or less of their degree through EMI experience linguistic 
gains? 

3. Is there a difference in linguistic gains between the two groups based on the amount 
EMI hours, immersion vs. semi-immersion? 

Materials and Methods 

The study took place at the Economics Department of a Catalan university, most of the 
participants were bilingual Spanish and Catalan speakers, and English was a third or additional 
language for them. The Economics department offers four undergraduate degrees, International 
Business (IBE), Economics (ECO), Management (MGMNT), and Business Management and 
Administration (ADMIN). All are four-year degree programmes. All courses are taught through 
EMI for the International Business degree (IBE), equalling 1500 EMI hours per academic year. 
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Whereas, the ECO, MGMT, and ADMIN programmes offer various intensities of EMI. Track 3 
and 4 offer no EMI courses in the first two years, while the second track offers 275 hours in the 
first year and 525 in the second year. The first track offers the most EMI courses equalling 625 
in the first year and 650 in the second year. In summary, the tracks range from offering 0 to 650 
hours of EMI per academic year. Figure 1 below shows a chart with the maximum amount of 
EMI hours per degree. In the case of this study, the classes can be considered to be EMI rather 
than ICLHE; there reportedly is no focus on form and no translation or dialogue in Catalan or 
Spanish. Lecturers provide all materials in English, such as power points and readings, 
according to information gathered from trainers on the programme. 


EMI Hours in the Economics Department 



□ IstYear B2ndYear □ 3rd Year D4thYear 


Figure 1. Chart illustrating EMI hours in the Economics department by degree. 

Design and Participants 

The study adopts a longitudinal pre-test post-test experimental design over one 
academic year, involving two data collection times and two separate groups of participants, 
immersion, henceforth IM, and semi-immersion, henceforth SIM. Data collection took place 
during the Fall and then again at the end of the academic year. Further details are omitted to 
preserve the anonymity of the data. 

The IM group: Participants (N=7) are first year IBE students (mean age=18, 2 female, 3 
male). L1= Spanish or Catalan. Their initial level of English was reported to be B1 or B2. 
Participants in the IM group have an exposure of 100% of their courses taught through EMI, 
which by the end of the academic year amounts to 1500 hours. 1 

The SIM group: Participants (N=9) first year ECO, MGMNT, or ADMIN students. Three 
female, six male, mean age=18. L1= Spanish or Catalan. Initial English level was reported to 


1 Due to the small number of students who returned for the second test, we have five individuals who completed both 
tests and four cross sectional participants, two females, two males, age 18. This was necessary to have similar 
numbers in each group and increase reliability of the statistics tests. Using two cross sectionals was not considered 
to alter the data due to the very similar profile of individuals. 
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be B1 or B2. This group has 18 to 41 percent of their courses taught through EMI, amounting in 
between 275 to 625 hours of EMI in the first year. All participants had a minimum of 275 EMI 
hours over the course of the year. 

Data Collection 

Students were recruited and asked to fill out an online questionnaire concerning 
language background, English abilities, past English language studies, and current exposure 
inside and outside the classroom. The participants were asked to complete a battery of tasks. 
The administration of the battery lasted two hours and took place outside of class time. There 
are two sections in the battery of tasks, a written component including three tests, listening 
comprehension, a cloze test, and a sentence manipulation task. The second section was an oral 
task. 2 


Listening task. The listening task is designed to measure participant’s aural abilities. An 
authentic live BBC radio interview is used; the instructions are given as follows: 

You will hear a recorded interview with the female group ETERNAL. Listen carefully to 
the recording and answer the following questions. You will have three minutes to study 
the questions before hearing the recording for the first time. You can start answering all 
questions immediately after the first listening. The recording will be played twice, and 
there will be another three minutes at the end for you to complete your answers. Now 
listen to the recording. 

The participants are then required to answer a series of fill in the blanks, multiple choice, 
and true and false questions. 

Lexico-grammatical tasks. In the cloze task, participants must fill in the blanks where 
there is only one correct lexical choice, measuring their lexico-grammatical abilities. The 
instructions are given as follows, the story contains twenty blanks: 


Fill EACH of the numbered blanks in the following passage with ONE suitable word. 

The lady who liked adventure 

It was one of those Impulse buys that can happen while shopping. Mary Bruce was in 

London looking for a nice dress . (1) she noticed a showroom with a light 

aircraft for . (2) at a terribly reasonable price. 

The grammar task consists of twenty sentence manipulations where the participant must 
change the structure of the sentence according to the prompt, this task is designed to measure 
syntactic abilities. The instructions are as follows: 

Instructions: 

Finish each of the following sentences in such a way that it is as similar as possible in 
meaning to the sentence printed before it. 

EXAMPLE: Despite Jack’s strange clothes, everybody ignored him. 

ANSWER: Nobody took . 

ANSWER: Nobody took notice of Jack’s strange clothes . 


2 Not analysed in this study. 
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Writing task. The written composition task requires the participant to write an essay in 
response to the following statement: 

In the space of the following 30 minutes write an essay about the following topic. Organize your 

ideas and make use of as wide a range of constructions and vocabulary as possible. 

Someone who moves to a foreign country should always adapt the customs and way of 

life of his/her new country, rather than holding onto his/her own customs. 

Writing an essay on a subjective topic leaves the participant free to write their opinions 
and thoughts requiring no specific content knowledge on the topic, while more advance students 
are free to write more complex compositions (Barquin, 2012). The writing task was analysed on 
three measures, Coordination, Accuracy, and Fluency (CAF), explained in more detail in the 
following sections. 

Analysis 

All data was evaluated quantitatively. The following sections describe the analysis of 
each task in turn. 

Written task. Written compositions were analysed using the Childes Computerized 
Language Analysis (CLAN) system. This was chosen to insure accurate non-biased results. 
The compositions were manually typed into word documents, the spelling and grammar auto- 
correct was disabled, and all errors were left untouched. Once formatted into CLAN files, the 
researcher coded each error according to the CAF measures (further explained below). Errors 
were coded as either grammatical, lexical, or pragmatic. Grammatical errors involved errors with 
function words, verb tense, prepositions, pronouns, and determiners. Lexical errors could be 
wrong lexical choice, lexical transfer, or idiosyncratic (creative non-words). Pragmatic errors 
included problems with referent (pronouns with no reference) incorrect use of idioms, 
expressions, or formulaic language. The compositions were also coded for correct use of 
discourse connectors or idiomatic expressions, for sentences and clauses (dependent, 
independent, or coordinating). See Appendix A for complete chart showing the codes used and 
brief description of each error. 

When assigning errors, the researcher tried to assign a single code to each error, the 
one that suited best. Although, it is possible that in some cases an error could be assigned two 
codes. Punctuation errors were not counted and spelling errors were only counted when they 
changed the word or resulted in another type of error. Inter-rater reliability was used where if 
there was a discrepancy it would be further analysed and agreement made between the 
researchers. 

Once coded, two tests were performed on the data using CLAN. The first, a frequency 
test used to calculate the number of times each of the codes appeared in a composition. The 
second, a test that calculates the number of lexical items used (tokens), the different types of 
lexical items used (types), and the ratio between the two, resulting in a Type/Token Ratio (TTR). 
All values were recorded and further values were calculated in Excel for CAF measures. The 
calculated ratios were then input into SPSS. Non-parametric Wilcoxon signed-ranks texts were 
used. The sample is quite small and this results in abnormally distributed data, so a parametric 
test could not be considered. The Wilcoxon test compares the means of two variables and 
detects if there is a significant difference between the two. In our case the results from the first 
and second test time were used for each measure. The result is significant if the P value is 
<.05. 


Linguistic Outcomes of English Medium Instruction Programmes in Higher Education 


57 





www.hlrcjournal.com 


Open 0 Access 


Complexity measures. Complexity measures involve two calculations, the first being the 
coordination index, calculating the number of coordinated clauses divided by total number of 
clauses minus sentences times 100, which gives a percentage of coordination. An increase in 
percentage signals an increased use of coordinated clauses, a decrease indicates increased 
use of dependent and subordinate clauses. Secondly, there is the Guiraud’s index of lexical 
diversity, which takes into account the length of text in regards to the TTR, and supplies a more 
balanced value for lexical complexity (Vermeer, 2000). These measures were adopted from 
Perez-Vidal (2014), previously used by Celaya & Perez-Vidal (2001), following Wolfe-Quintero, 
Inagaki, and Kim (1998). 

Accuracy measures. Accuracy measures involve calculating errors per word, and errors 
per clause, based on the errors that were coded into the CLAN system. As explained, the errors 
could be lexical, grammatical, or pragmatic in nature. The result of this measure is expressed in 
a numeric value and describes the rate of accurate lexical choice, verb use, and correct 
morphology. As the value decreases it signifies that fewer errors are detected. 

Fluency measures. The measure for fluency is a calculation of words per clause and 
words per sentence, following the concept that longer clauses and sentences are more fluent 
than shorter simpler clauses and sentences. To obtain this value the count test was performed 
on the data using the CLAN programme as outlined before. 

Listening task. The listening tasks were corrected manually and errors were recorded 
in an Excel table. One point was awarded for a correct response and no point was awarded for 
an incorrect response. The score from test one was subtracted from the score from test two in 
order to obtain the amount of gain for each participant. This data was then transferred to SPSS 
for further statistical analysis. 

Lexico-grammatical tasks. The cloze and grammar tasks were corrected in order to 
obtain qualitative data. The correction method assumed one correct answer for each question, 
so one point was given for a correct response and no point was given for an incorrect response. 
As for the listening the value of the gain was calculated by subtracting the first test score from 
the second and the data was analysed in SPSS. 

Results 

The results of the tests were examined first by looking at the descriptive statistics to see 
if there were any overall mean differences on each task between time one (T1) and time two 
(T2). Results found are broken down by task: listening, lexico-grammatical, and writing. After 
reviewing the descriptive statistics, the non-parametric Wilcoxon signed ranks test was carried 
out to detect if there were any significant gains between the two tests, for each of the tasks 
examined. 

Listening Task 

Descriptive statistical analysis of the listening task, as shown in Table 1, reveals a lower 
value on T2 for both IM and SIM groups, by 1 point. This signifies a weaker performance on the 
listening task, at the second examination period. 
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Table 1. Listening Task 


Mean Value 

Time 1 

Time 2 

Immersion (N=7) 

58.9 

57.1 

Semi Immersion (N=9) 

49.5 

48.1 


Lexico-Grammatical Tasks 

The results of the lexico-grammar task displayed in Table 2 reveal that means of the 
grammar task show an improvement of 12.6 points from T1 to T2 for the IM group. While the 
SIM group shows an average improvement of 11.6. This is a positive result and signals a trend 
for improvement in grammar tasks. 


Table 2. Grammar Task 


Mean Value 

Time 1 

Time 2 

Immersion (N=7) 

44.5 

57.1 

Semi Immersion (N=9) 

32.0 

43.6 


Table 3 shows that the means from the cloze task also point towards average 
improvements, a greater improvement for the IM group going up 14.1 points from 33.4 to 47.5 
and a slight improvement for the SIM group from 32.0 to 32.3. 


Table 3. Cloze Task 


Mean value 

Time 1 

Time 2 

Immersion (N=7) 

33.4 

47.5 

Semi Immersion (N=9) 

32.0 

32.3 


Table 4. Complexity , Accuracy , and Fluency (CAF) Measures in Immersion Group 


Immersion Group 

Min. 

Max. 

Mean 

Stand. Dev. 

Guiraud’s index 



Time 1 

6.60 

9.03 

7.61 

.90 

Time 2 

6.55 

9.13 

7.90 

.99 

Coordination index 


Time 1 

.00 

160.00 

75.03 

63.71 

Time 2 

.00 

166.67 

38.03 

60.60 

Words per Sentence 


Time 1 

16.71 

27.38 

20.71 

4.23 

Time 2 

16.58 

36.43 

22.25 

6.61 

Words per Clause 


Time 1 

11.50 

16.85 

13.24 

2.26 

Time 2 

9.25 

28.33 

14.83 

6.42 
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Errors per Word 




Time 1 

.06 

.16 

.10 

.03 

Time 2 

.06 

.12 

.09 

.02 

Errors per Clause 




Time 1 

.80 

1.85 

1.31 

.42 

Time 2 

.88 

1.78 

1.25 

.41 


Number 7 


The analysis of the IM group’s writing task yields the following results, as displayed in 
Table 4. For complexity, a slight improvement on the Guiraud’s index (<1.) is found. There is a 
decrease of 37 in the coordination index, suggesting more subordinate (dependent) and 
independent clauses used at T2. In regards to fluency measures, there is a 1-point increase in 
both words per sentence and words per clause. Finally, for accuracy measures there is a slight 
improvement (<.1), as less errors were detected in the calculation for errors per word or errors 
per clause at T2. 


Table 5. Complexity, Accuracy, and Fluency (CAF) Measures in Semi-Immersion Group 


Semi-Immersion Group Min. 

Max. 

Mean 

Stand. Dev. 

Guiraud’s index 

Time 1 6.33 

9.69 

7.54 

.94 

Time 2 7.89 

9.43 

8.35 

.54 

Coordination index 

Time 1 .00 

300.00 

86.65 

88.35 

Time 2 .00 

133.33 

52.74 

47.28 

Words per Sentence 

Time 1 18.33 

23.73 

19.96 

1.92 

Time 2 15.43 

31.86 

23.01 

6.64 

Words per Clause 

Time 1 9.96 

16.25 

12.76 

1.99 

Time 2 8.53 

15.19 

11.95 

2.27 

Errors per Word 




Time 1 .08 

.19 

.13 

.04 

Time 2 .09 

.17 

.13 

.02 

Errors per Clause 




Time 1 .92 

2.56 

1.66 

.64 

Time 2 .83 

2.13 

1.52 

.40 


Number 9 


A similar trend is noted in the SIM group, as shown in Table 5. There is a higher mean 
value on the Guiraud’s index at T2 (<1.). A 33.91 difference between T1 and T2 was measured 
on the coordination index, suggesting more use of independent and subordinate (dependent) 
clauses. Regarding fluency, there is a 3-point difference between the two means for words per 
sentence, suggesting longer sentences in the second test. However, when looking at words per 
clause, the mean decreases slightly there (<1.). On accuracy, measures the means remain 
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virtually the same for both T1 and T2, suggesting almost no change in regards to errors per 
word or errors per clause, for the SIM group. 

Wilcoxon Signed-Ranks Test 

In order to detect if the improvements noted on examination of the descriptive statistics 
were in fact statistically significant the Wilcoxon signed ranks test was performed on the data 
and displayed in Tables 6 and 7. This test was selected to compare the difference of mean 
values from T1 to T2 on each of the tasks measured. 


Table 6. Immersion Group 


Median Differences Between Test One and Test Two 

Wilcoxon Signed Rank Test Significance Value 

Listening 

.10 

Grammar 

.79 

Cloze 

.50 

Fluency 

1.0 words per clause 

.87 words per sentence 

Accuracy 

.31 errors per word 

.87 errors per clause 

Complexity 

.24 coord, index 

.61 Guiraud’s index 


Note. Value is significant if p=<05. 


The results for the first three tasks, listening grammar, and cloze, all reflect values where 
p=>.05 (Table 6). The CAF measures also reflect values where p= >.05, for all tasks. The 
interpretation is that the IM group shows no significant gains in any of the language domains 
measured. 


Table 7. Semi-immersion group 


Median Differences Between Test one and Test Two 

Wilcoxon Signed Rank Test Significance Value 

Listening 

.77 

Grammar 

. 01 * 

Cloze 

.82 

Fluency 

.14 words per clause 

.21 words per sentence 

Accuracy 

.95 errors per clause 

.58 errors per clause 

Complexity 

.06 coord, index 

.05 Guiraud’s index 


Note. Value is significant if p=< .05. 
‘Significant 


The SIM group results (Table 7) show no significant improvements for the cloze or 
listening task, between T1 and T2. A significant gain was detected in the grammar task (p=.01). 
Regarding the CAF measures there were non-significant results for fluency and accuracy. 
However, the Guiraud’s index (p=.05) is very close to a significant finding. 
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Discussion 

Referring back to research question one (do students who take their entire degree 
through EMI experience linguistic gains over a one-year period?), results for the IM group, who 
receive their entire degree programme through English were positive. Mean gains in lexico- 
grammatical tasks were observed, and also on all CAF measures on the writing task. No 
improvements were noted on the listening task. Although the means pointed towards an 
improvement when tested, statistically significant gains were not detected using non-parametric 
tests. Thus, it can be asserted that there is a trend for improvement in lexico-grammatical and 
writing tasks in students who study their entire degree programme through EMI. 

Regarding the second research question (do students who take less than half of their 
degree through EMI experience linguistic gains?), the results for the SIM group point to a trend 
towards improvement on all tasks save the listening task. Further, when analysed, statistically 
significant linguistic gains were detected in both lexico-grammatical tasks as well as lexical 
complexity in the writing task. These findings suggest students enrolled in semi immersion EMI 
courses do experience linguistic gains in the domains of complexity of writing and grammar, as 
well as a general trend towards linguistic improvement. 

Addressing the third research question (is there a difference in linguistic gains between 
the two groups based on the type of programme; that is, the amount of EMI hours, immersion 
vs. semi-immersion?), the data point towards similar mean gains from T1 to T2 in both the IM 
and the SIM groups. The mean values reflect more average improvements in the IM group but 
when tested statistically significant gains were detected in the SIM group only. 

Some reasons as to why this could be is that participants have achieved a B1 or B2 level 
in English and may have plateaued. Without putting a lot of effort and having some formal 
instruction, many adults experience difficulty reaching advanced levels in a second language. 
Students may feel they have a good enough level and that they can get by, giving them little 
incentive to enhance their English language skills, as Wilkinson (2004) stated, “language is a 
tool once they have the minimum needed there is little incentive to enhance skills” (p. 453). 

The second factor that may inhibit linguistic gains is the question of input, and 
interaction. As we saw earlier from Tazl’s (2011) and Airey & Linder (2004) students as well as 
lecturers behave slightly different in EMI classes. Students tend to interact less and ask less 
questions, also lecturers stick to the plan more and tend to follow the slides, with less interaction 
storytelling and other such nuances. This may affect the amount of interaction with English and 
therefore their output abilities. In addition, many professors are lecturing in their second or 
additional language and are not language specialists. Thus, there is little or no correction of 
students’ errors in class or in assignments; instead, focus is entirely on content so students 
have little feedback on their language skills. As Sercu (2004) summed up: 

If lecturers cannot provide appropriate language input, if students are not provided with 
adequate opportunities for interaction in the foreign language, if students do not already 
possess a command of the language that allows them to benefit from English-medium 
instruction, the hoped for enhancement of students’ language skills may remain forth¬ 
coming. (p. 548) 

Some limitations of the study and suggestions for further research are: the current study 
was a preliminary study and it would be well worth it to repeat the same analysis on a larger 
number of students. It would be interesting to measure the students over a longer period of 
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time, their entire degree programme perhaps. It would be interesting to observe the lectures in 
order to have a clear picture as to exactly what kinds of interactions are going on and exactly 
what the input is. 


Conclusion 

This preliminary study has provided a background and review of the relatively new field 
of EMI in the European context. The study examined the linguistic outcomes of two groups of 
EMI undergraduate students in a Catalan university. Findings show a trend towards 
improvement for both groups of students measured through a listening task, two lexico- 
grammatical tasks, and a composition task where accuracy complexity and fluency were 
measured. A significant gain in grammar skills is evidenced in the semi-immersion (SIM) group, 
but not in the immersion group (IM), which only shows a tendency toward improvement. 
Regarding the lack of progress in the listening test, the lecturers are non-native speakers of 
English. Due to this, the students do not have native accent input and this could be a factor as 
to why their listening comprehension of native speakers does not improve. However, they 
improve on grammar of which they receive more target like input through readings, 
presentations and lectures. 

This preliminary study contributes to the field by highlighting a trend for linguistic 
improvement after EMI treatment. However, further research is needed to confirm the trend 
noted. 
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Appendix A 


Grammatical 
Error Codes 

All types were counted as grammatical errors 

GrEitrans 

Grammatical error as a result of LI transfer, using a wrong structure due to LI 
transfer. 

GrE:fw 

Inappropriate use of a function word (pronouns, determiners, prepositions, 
conjunctions) or omission in obligatory contexts. 

GrE:adv/j 

Comparatives and superlatives, when an adverb/adjectives used inappropriately, or 
in the wrong form. 

GrE:n 

Missing noun, wrong form (morphology). 

GrE:v 

Tense, aspect, auxiliary verb missing, omission of a verb that is needed in the 
sentence structure, wrong form (morphology). 

GrEiwo 

Error due to wrong word order, one or several elements that are misplaced in the 
sentence, lack of inversion in questions. 

GrE:ag 

Lack of agreement between subject and verb or between determiner and noun. 

GrEineg 

Errors with negatives, double negatives, confusion between no/not, negative particle 
errors. 

Lexical Error 
Codes 

All types were counted as lexical errors 

LexE:idio 

Idiosyncratic form used by the learner, creative morphology or made up non-words. 

LexE:trans 

Words directly borrowed from the LI whether modified or not, false friends. 

LexE:cho 

Wrong lexical choice, mistakes with commonly confused words, use of words in 
inappropriate text. 

LexE:omi 

Omission of a lexical item, the learner cannot retrieve. 

Pragmatic 
Error Codes 

All were counted as pragmatic errors 

PragEiref 

Erroneous use of reference markers, ambiguous references. 

PragEiidio 

Idiosyncratic usage not clearly ungrammatical, problems with formulaic language and 
idioms. 

PragEicon 

Wrong discourse connector. 

F: 

Discourse connectors of the kind ‘to sum up’, ‘for example' and idiomatic expressions 
used correctly. 

Syntactic 

Complexity 


S 

Sentence, any string between full stops (as the writer has written it). When writers 
miss punctuate the researcher identifies sentences according to syntactic structures. 

C 

Independent clauses. 

DepC 

Dependent clauses. 

Coord 

Coordinating clauses. 
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